Support Vector Machine Lagrange Multipliers and Simplex Volume Decompositions

The Support Vector Machine (SVM) idea has attracted recent attention in solving classiication and regression problems. As an example based method, SVMs distinguish two point classes by nding a separating boundary layer, which is determined by points that become known as Support Vectors (SVs). While the computation of the separating boundary layer is formulated as a linearly constrained Quadratic Programming (QP) problem, in practice the corresponding dual problem is computed. This paper investigates how the solution to the dual problem depends on the geometry. When examples are separable, we will show that the Lagrange multipliers (the unknowns of the dual problem) associated with SVs can be interpreted geometrically as a normalized ratio of simplex volumes, and at the same time a simplex volume decomposition relation must be satissed. Examples for the two and three dimensional cases are given during the discussion. Besides showing geometric properties of SVMs, we also suggest a way to investigate the distribution of the Lagrange multipliers based on a random matrix model. We nish this paper with a further analysis of how the Lagrange multipliers depend on three critical angles using the Singular Value and CS decompositions.